Skip to content

Conversation

@qdm12
Copy link
Contributor

@qdm12 qdm12 commented Jan 7, 2021

  • Shell entrypoint to run sed command using the environment variable WEBSOCKET_HOST and defaulting to localhost if it's not set. It then executes /bin/sh.

  • Faster Docker builds by caching node modules in a Docker layer for build

  • Pin Alpine and Node versions

  • Smaller final Docker image with alpine

  • Run without root as user with uid 1000, for security reasons 👀 - will do with the Go entrypoint due to how nginx operates

  • ❓ We should have an http server in the Dockerfile, do you have any preference? Maybe nginx or a Go binary would be nice to keep the size lower than a node image with serve. I'm happy to hack one with Go, or you can use my ugly 1 line one (lol) from this

  • Add documentation once we're all happy

qdm12 added 2 commits January 6, 2021 19:54
- Pin Alpine and Node versions
- Use Alpine for final image (smaller)
- Cache node modules in a Docker layer for build
- Run without root as user with uid 1000
@GRVYDEV
Copy link
Owner

GRVYDEV commented Jan 7, 2021 via email

@qdm12
Copy link
Contributor Author

qdm12 commented Jan 7, 2021

I’d be interested in a Go binary though because as of right now Nginx output

Cool, I'll write an http server this weekend to serve this react code. If you don't mind, I'll add some features to modify config.json as well so it can replace the entrypoint.sh at the same time. With such server, we can then easily cross CPU compile it so it works on all CPU architectures supported by Docker as well (arm, 386, s390x etc.).

@Woodham
Copy link
Contributor

Woodham commented Jan 7, 2021

One small concern but worth pointing out - the swd command will run every time the container is started, but if the env var is changed for whatever reason then it will fail to update the config file on subsequent runs, and would require destroying the container entirely.

@qdm12
Copy link
Contributor Author

qdm12 commented Jan 7, 2021

@Woodham we can copy the files to a temp directory and modify them there at every start, and clear the directory at start or shutdown. Or use a regex so that it replaces the entire line for example. I think the temp directory may be better for ease of modifications.

@GRVYDEV
Copy link
Owner

GRVYDEV commented Jan 7, 2021

@Woodham would mounting the config file fix this issue? Im not sure running the sed command is the solution we want to go with long term because we will have more configuration in the future than just wsurl

@qdm12
Copy link
Contributor Author

qdm12 commented Jan 7, 2021

Why not offer both solutions 😉 We could have the original source in /lightspeed/original, have a /lightspeed/override directory and an entrypoint which would take care of having files in override take precedence over env variables. Although that would mean more maintenance for the entrypoint.

will have more configuration in the future than just wsurl

Are you talking about ten parameters or more a thousand?

@GRVYDEV
Copy link
Owner

GRVYDEV commented Jan 7, 2021

Why not offer both solutions wink We could have the original source in /lightspeed/original, have a /lightspeed/override directory and an entrypoint which would take care of having files in override take precedence over env variables. Although that would mean more maintenance for the entrypoint.

will have more configuration in the future than just wsurl

Are you talking about ten parameters or more a thousand?

closer to 10 than 1000

@Woodham
Copy link
Contributor

Woodham commented Jan 7, 2021

If you'd prefer to stick to env vars (and I totally understand why) you could create a separate config.json.template file and don't do the sed replacement in-line.

@GRVYDEV
Copy link
Owner

GRVYDEV commented Jan 11, 2021

Yeah I think we want to avoid a sed command here. I think the best thing would be to provide a template config.json and then mount it inside the docker container

@qdm12
Copy link
Contributor Author

qdm12 commented Jan 11, 2021

Personally I'd go against letting the user manage files, as it's prone to error vs environment variables. For example: windows vs Unix line endings, permission issues, bad formatting of JSON, more tedious debugging. It's also a few extra steps to get it working.

Anyway let me know what you think guys, I'll get started on an http server entrypoint soon.

@Woodham
Copy link
Contributor

Woodham commented Jan 11, 2021

Hey @qdm12 - I had a bit of a think about this and I think the best way to get this going for now is to do something similar to the way the nginx dockerfile approaches this problem. It uses a utility called envsubst to replace placeholders in a file with environment variables.

I've opened a PR (to your PR branch) with an implementation of this if you'd like to take a look (plus a couple of other changes). - https://github.com/qdm12/Lightspeed-react/pull/1

* Change final base image to nginx
* use envsubst to replace environment variables in config file
* create config file template
* move docker specific files to directory
@qdm12
Copy link
Contributor Author

qdm12 commented Jan 12, 2021

@Woodham PR is merged into mine 👍 Thanks!

I think this is good, everyone's happy for now? I'll add documentation if so.

I'll do a Go binary entrypoint / scratch based image in a subsequent PR.

@Woodham
Copy link
Contributor

Woodham commented Jan 12, 2021

Hopefully it all makes sense to you @GRVYDEV. I think this should be enough to allow people to easily deploy an instance :)

@GRVYDEV
Copy link
Owner

GRVYDEV commented Jan 12, 2021

Hopefully it all makes sense to you @GRVYDEV. I think this should be enough to allow people to easily deploy an instance :)

Yeah Im not sure this makes much sense to me right now. Here is the solution I came up with for docker compose. I guess the only downside to my solution is that it doesn't work outside of compose. Im a bit confused at how this solution works. So at runtime the entrypoint.sh file is run which copies the config.json.template to the publicdir/config.json and the values in the config.json.template file are replaced by the env vars?

@Woodham
Copy link
Contributor

Woodham commented Jan 12, 2021

Hopefully it all makes sense to you @GRVYDEV. I think this should be enough to allow people to easily deploy an instance :)

Yeah Im not sure this makes much sense to me right now. Here is the solution I came up with for docker compose. I guess the only downside to my solution is that it doesn't work outside of compose. Im a bit confused at how this solution works. So at runtime the entrypoint.sh file is run which copies the config.json.template to the publicdir/config.json and the values in the config.json.template file are replaced by the env vars?

Yep you've got it exactly - it uses a command line tool called envsubst to replace env var placeholders in a template with environment variables.

I actually agree that the mounting the config file like you've done there is the simplest solution (and can definitely work outside of compose btw - you can mount files like that using the normal docker run command too) - the benefit here primarily that env vars can be used instead. I think which way to go is purely preference at this point.

This specific implementation currently takes advantage of the fact that the nginx docker image will run any .sh files put into a specific folder on startup automatically, which is why it doesn't need to be set manually as the entrypoint or cmd - nginx is.

@qdm12
Copy link
Contributor Author

qdm12 commented Jan 30, 2021

@Woodham

I think which way to go is purely preference at this point.

It's also about abstraction. Having as environment variables allows the underlying implementation (nginx, react's config.json) to change. And implementations do change over time, so it might be a good idea not to rely on the user bind mounting a config.json file.

@qdm12
Copy link
Contributor Author

qdm12 commented Jan 30, 2021

I think we should go forward and merge this with that environment variable substitution.

I would like to contribute with a Go http server & env variables "entrypoint" program but that's a bit blocked by this PR first.

@qdm12
Copy link
Contributor Author

qdm12 commented Jan 30, 2021

@GRVYDEV also are we planning on having this frontend served as a microservice (have its own image part of a docker-compose/k8s)? If so I'd need you to:

  • give me your Docker Hub username
  • what image name you would like for it (lightspeed-react maybe?)
  • set your Docker Hub password as a Github secret with key DOCKERHUB_PASSWORD

I can set it up to build and push to Docker Hub using Github Actions for all cpu architectures (raspberry pis and so on).


```sh
docker build -t grvydev/lightspeed-react https://github.com/GRVYDEV/Lightspeed-react.git
```
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that steps 1 and 2 will be removed once the image is built and pushed by the CI to Docker Hub

@nitrag
Copy link

nitrag commented Jan 31, 2021

Here is my WIP docker-compose.yaml file.

.env

# IP could be your computer in development or Public IP in production
IP_ADDRESS=192.168.156.180
WEB_PORT=8888
INGEST_PORT=8084
WEBSOCKET_HOST=localhost
WEBSOCKET_PORT=8889

docker-compose.yaml

version: '3'

services:
  lightspeed-ingest:     
    restart: on-failure
    build: 
      context: ../lightspeed-ingest 
      dockerfile: Dockerfile
      #target: lightspeed-ingest
    image: lightspeed-ingest
    env_file: '.env'
    ports:
      - "${INGEST_PORT}:8084"
  
  lightspeed-react:    
    restart: on-failure
    image: lightspeed-react
    build:
      context: ../Lightspeed-react
      dockerfile: Dockerfile
      #target: lightspeed-react
    env_file: '.env'
    ports:
      - "${WEB_PORT}:80"

  lightspeed-webrtc:     
    restart: on-failure
    build: 
      context: ../Lightspeed-webrtc
      dockerfile: Dockerfile
      #target: lightspeed-webrtc
    image: lightspeed-webrtc
    env_file: '.env'
    command: ["lightspeed-webrtc", "--addr=0.0.0.0", "--ip=${IP_ADDRESS}", "--ports=20000-20500", "run"]
    ports:
      - ${WEBSOCKET_PORT}:8080 # WebRTC
      - 65535:65535/udp # RTP
      - 20000-20500:20000-20500 # WebRTC PeerConnection
      - 20000-20500:20000-20500/udp # WebRTC PeerConnection UDP

I'd personally prefer to not have to use host network and specify ports explicitly. But I don't have it working because I don't understand the required tcp/udp ports yet. I'm not getting any errors or logs, OBS says it's streaming.

The build: context: is the location of the other repositories and their Dockerfiles if you want to build from source. Or you can comment out build: and just use image: to be the DockerHub repository name/tag.

For building locally we can do Git Submodules or the alternative:

  1. User creates a folder on the computer ~/SomeFolder/
  2. User clones Project-Lightspeed repo to get the docker-compose, Readme, documentation, etc.
  3. User clones each lightspeed repo (ingest, react, webrtc) via git clone ....
  4. User goes to ~/SomeFolder/Project-Lightspeed
  5. Runs docker-compose up

I have a slight hesitation in using Git Submodules because that might make it harder/tedious for the developers who need to checkout and work on different branches. I honestly haven't implemented submodules before so I could be wrong, just keep that in mind.

@qdm12
Copy link
Contributor Author

qdm12 commented Jan 31, 2021

For building locally we can do Git Submodules or the alternative

Ideally maybe we can keep it to separate repos and the user can just docker-compose pull the 3 images in a docker-compose.yml? We could also specify a build context using for example

build: https://github.com/GRVYDEV/Lightspeed-react.git

For each image, so the user/dev can just docker-compose build to build all the images from each of their repositories, provided they have docker, docker-compose and git installed. I'm not sure we really need to use git submodules or have them all in the same directory, at least if we focus on Docker usage. Using it without Docker is quite a pain though (build things locally and all for each repo).

@nitrag
Copy link

nitrag commented Jan 31, 2021

@qdm12 Yep see this:

GRVYDEV/Project-Lightspeed#34

@GRVYDEV I'm good with this PR.

Copy link
Owner

@GRVYDEV GRVYDEV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good

@GRVYDEV GRVYDEV merged commit e937aa9 into GRVYDEV:master Feb 8, 2021
@qdm12 qdm12 deleted the dockerfile-improvements branch February 9, 2021 02:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants